An Analytic Method for Evaluating the Performance of Aggregation Rules for Probability Densities

Abstract It is shown how infinite sequences of densities with defined properties can be used to evaluate the expected performance of mathematical aggregation rules for elicited densities. The performance of these rules is measured through the expected variance, calibration, and expected Brier score of the aggregate. A general result for the calibration of the arithmetic average of densities from well-calibrated independent experts is given. Arithmetic and geometric aggregation rules are compared in several demonstrations using sequences of uniform, normal, and exponential densities. Sequences are developed that exhibit dependence among experts and lack of calibration. The impact of correlation, number of experts, and degree of calibration on the performance of the aggregation is demonstrated with normal densities.